Mounting device for maintaining rigid alignment between cameras

ABSTRACT

A mounting device includes an elongated beam having a first end portion, a second end portion, and a side surface extending between the first end portion and the second end portion. The mounting device also includes a first camera mount attached to the first end portion configured to support a first camera, a second camera mount attached to the second end portion configured to support a second camera, and a bracket for fixedly connecting the elongated beam to a vehicle. The bracket is positioned between the first end portion and the second end portion. The bracket includes at least one base configured to be attached to the vehicle and a wall extending from the at least one base comprising an opening sized to receive the elongated beam, such that engagement between the wall and the elongated beam restricts rotation of the elongated beam about multiple axes.

TECHNICAL FIELD

This disclosure relates generally to a mounting bracket or device forcameras and, in particular, to a mounting bracket or device configuredto maintain proper rigid alignment between cameras to capture imagesthat can be used to create stereo images of a scene. The mountingbracket or device can be configured for use with autonomous orsemi-autonomous vehicles.

BACKGROUND

Accurate and consistent obstacle detection and navigation can be keyelements of autonomous or semi-autonomous driving applications.Typically, an autonomous or semi-autonomous vehicle utilizes variouson-board sensors to detect obstacles, other aspects of the roadway,and/or other aspects of an environment around the vehicle, which can bereferred to as “perception information” or “perception data”representing what an ordinary driver would perceive in the surroundingenvironment of a vehicle. Examples of such sensors include one or moreof vision sensors (e.g., camera(s)), radio detection and ranging (e.g.,radar) sensors, and/or light detection and ranging (e.g., LiDAR)sensors. The perception information detected by the on-board sensors isprocessed and analyzed by image analysis software or a perception systemto identify the objects surrounding the vehicle. The objects mayinclude, for example, signaling devices, such as traffic lights, roadwayboundaries, other vehicles, pedestrians, and/or obstacles.

Autonomous or semi-autonomous vehicles can include vision sensors orcameras configured to obtain stereo or three-dimensional images, whichinclude information about distances between objects in a scene and/orinformation about a depth or distance between the cameras and theobjects. Typically, a stereo camera assembly includes two or morecameras mounted to a portion of the autonomous vehicle. For example, twocameras may be positioned a distance apart, pointing in the samedirection, and can be carefully aligned with each other to generatethree-dimensional distance data. Obtained distance data may be used byvarious algorithms, such as the vehicle perception system, to helpdetect or identify objects, as well as for vehicle navigation. However,because stereo cameras rely on a tight, rigid alignment between the twoor more cameras, which needs to be known by the software ahead of time,perception information obtained from these type of stereo cameraassemblies may not be reliable when the vehicle is in motion and/or isexposed to certain environmental conditions. In particular,environmental conditions may affect alignment of and/or distance betweenthe cameras reducing accuracy of distance information determined by theperception system. For example, in an autonomous vehicle application,while the vehicle is on the road, the stereo cameras may go out ofalignment due to vibrations and/or due to thermal expansion orcontraction (i.e., thermal shock) of portions of the vehicle or cameramount assembly. Distance information determined by the perception systemfrom analysis of captured stereo images becomes more unreliable asmisalignment of the cameras increases. In some cases, misalignment ofthe cameras of a stereo camera assembly can be addressed by frequentlycalibrating the cameras or through image processing to detect andaccount for any misalignment between the cameras. However, frequentcalibration or image processing routines may be difficult to implementfor certain vehicle perception systems.

SUMMARY

In order to address such issues, the mounting devices, image systems,vehicles, and methods of the present disclosure are configured topreserve alignment between cameras of a stereo camera assembly orsystem, thereby avoiding the need for frequent calibration or softwareprocessing routines to compensate for camera misalignment. Further, themounting devices, image systems, vehicles, and methods of the presentdisclosure are configured for use with autonomous and semi-autonomousvehicles so that stereo or three-dimensional image processing techniquescan be used to obtain distance information that can be used by existingvehicle perception systems to detect and identify objects in a scene andfor vehicle navigation.

According to an aspect of the disclosure, a mounting device includes anelongated beam having a first end portion, a second end portion, and aside surface extending between the first end portion and the second endportion. The mounting device also includes: a first camera mountattached to the first end portion configured to support a first camera;a second camera mount attached to the second end portion configured tosupport a second camera; and a bracket for fixedly connecting theelongated beam to a vehicle. The bracket is positioned between the firstend portion and the second end portion. The bracket includes at leastone base configured to be attached to the vehicle and a wall extendingfrom the at least one base comprising an opening sized to receive theelongated beam, such that engagement between the wall and the elongatedbeam restricts rotation of the elongated beam about multiple axes.

According to another aspect of the disclosure, a system includes amounting device. The mounting device includes an elongated beam having afirst end portion, a second end portion, and a side surface extendingbetween the first end portion and the second end portion. The elongatedbeam also includes a first camera mount attached to the first endportion configured to support a first camera, a second camera mountattached to the second end portion configured to support a secondcamera, and a bracket positioned between the first end portion and thesecond end portion for fixedly connecting the elongated beam to avehicle. The bracket includes at least one base configured to beattached to the vehicle and a wall extending from the at least one basehaving an opening sized to receive the elongated beam, such thatengagement between the wall and the elongated beam restricts rotation ofthe elongated beam about multiple axes. The mounting device is fixedlyconnected to an exterior of a vehicle body by one or more fastenersextending through openings of the at least one base of the bracket ofthe mounting device. The system also includes a first camera attached tothe first camera mount, a second camera attached to the second cameramount, and at least one processor in electrical communication with thefirst camera and the second camera. The at least one processor isconfigured to generate at least one stereo image of a scene based on afirst image received from the first camera and a second image receivedfrom the second camera, wherein the first image and the second image areacquired substantially simultaneously.

According to another aspect of the disclosure, a mounting deviceincludes an elongated beam formed from a first material and having afirst end portion, a second end portion, and a side surface extendingbetween the first end portion and the second end portion. The mountingdevice further includes: a first camera mount attached to the first endportion configured to support a first camera; a second camera mountattached to the second end portion configured to support a secondcamera; and a bracket formed from a second material positioned betweenthe first end portion and the second end portion for fixedly connectingthe elongated beam to a vehicle. The bracket includes at least one baseconfigured to be attached to the vehicle and a wall extending from theat least one base having an opening sized to receive the elongated beam.The side surface of the elongated beam is adhered to an inner surface ofthe wall by an adhesive. The adhesive includes a cured epoxy and aplurality of spacers embedded in the cured epoxy for maintainingsubstantially equal spacing between the side surface of the beam and theinner surface of the wall.

According to another aspect of the disclosure, a method for attaching afirst camera and a second camera to a vehicle includes inserting anelongated beam of a mounting device through an opening in a wall of abracket of the mounting device, such that a portion of the elongatedbeam between a first end portion and a second end portion of theelongated beam is retained in the opening. The method also includesattaching the bracket of the mounting device to the autonomous orsemi-autonomous vehicle by securing one or more fasteners throughopenings in at least one base of the bracket to fixedly connect thebracket to the vehicle. The method also includes attaching the firstcamera to a first camera mount of the mounting device and the secondcamera to a second camera mount of the mounting device, therebystabilizing the first camera and the second camera a fixed distanceapart so that stereo images of the scene can be obtained by the firstcamera and the second camera.

BRIEF DESCRIPTION OF THE DRAWINGS

Additional advantages and details are explained in greater detail belowwith reference to the exemplary embodiments that are illustrated in theaccompanying schematic figures, in which:

FIG. 1A is a schematic drawing of an exemplary autonomous vehiclesystem, according to an aspect of the present disclosure;

FIG. 1B is a side view of an autonomous or semi-autonomous vehicleincluding a sensor frame or housing on the roof of the vehicle,according to an aspect of the present disclosure;

FIG. 2 is a schematic drawing illustrating exemplary system architecturefor an autonomous or semi-autonomous vehicle, according to an aspect ofthe present disclosure;

FIG. 3A is a perspective view of a front portion of a mounting devicefor alignment of cameras, according to an aspect of the disclosure;

FIG. 3B is a perspective view of a rear portion of the mounting deviceof FIG. 3A;

FIG. 3C is a front view of the mounting device of FIG. 3A;

FIG. 3D is a side view of the mounting device of FIG. 3A;

FIG. 3E is a front view of a camera mount of the mounting device of FIG.3A;

FIG. 3F is a perspective view of an elongated beam of the mountingdevice of FIG. 3A;

FIG. 3G is a cross-sectional view of the elongated beam of FIG. 3F takenalong line 3G;

FIG. 3H is a perspective view of a bracket of the mounting device ofFIG. 3A;

FIG. 3I is a side view of the bracket of FIG. 3H;

FIG. 3J is a front view of the mounting device of FIG. 3A with camerasattached to the camera mounts;

FIG. 4A is a schematic drawing of a mounting device for alignment ofcameras attached to a vehicle, according to an aspect of the presentdisclosure;

FIG. 4B is a schematic drawing of the mounting device of FIG. 4A whenexposed to thermal expansion and/or vibration forces;

FIG. 4C is a schematic drawing showing an example of a mounting devicefor alignment of cameras including vibration damper assemblies,according to an aspect of the present disclosure;

FIG. 4D is a schematic drawing of the mounting device of FIG. 4C whenexposed to thermal expansion and/or vibration forces;

FIG. 5 is a flow chart showing steps for assembling a mounting devicefor alignment of cameras on a vehicle, according to an aspect of thedisclosure;

FIG. 6 is a system for generating stereo images from cameras connectedto a mounting device that aligns cameras of a vehicle, according to anaspect of the disclosure; and

FIG. 7 is a schematic drawing of components of a computer system thatcan be used with an autonomous or semi-autonomous vehicle, according toan aspect of the disclosure.

DETAILED DESCRIPTION

The following description is provided to enable those skilled in the artto make and use the described embodiments contemplated for carrying outaspects of the disclosure. Various modifications, equivalents,variations, and alternatives, however, will remain readily apparent tothose skilled in the art. Any and all such modifications, variations,equivalents, and alternatives are intended to fall within the spirit andscope of the present disclosure.

For purposes of the description hereinafter, the terms “upper”, “lower”,“right”, “left”, “vertical”, “horizontal”, “top”, “bottom”, “lateral”,“longitudinal”, and derivatives thereof shall relate to the disclosureas it is oriented in the drawing figures. However, it is to beunderstood that the disclosure may assume alternative variations andstep sequences, except where expressly specified to the contrary. It isalso to be understood that the specific devices and processesillustrated in the attached drawings, and described in the followingspecification, are simply exemplary embodiments of the disclosure.Hence, specific dimensions and other physical characteristics related tothe embodiments disclosed herein are not to be considered as limiting.

No aspect, component, element, structure, act, step, function,instruction, and/or the like used herein should be construed as criticalor essential unless explicitly described as such. Also, as used herein,the articles “a” and “an” are intended to include one or more items, andmay be used interchangeably with “one or more” and “at least one.” Asused herein, the terms “has,” “have,” “having,” or the like are intendedto be open-ended terms. Further, the phrase “based on” is intended tomean “based at least partially on” unless explicitly stated otherwise.

As used herein, the term “communication” may refer to the reception,receipt, transmission, transfer, provision, and/or the like, of data(e.g., information, signals, messages, instructions, commands, and/orthe like). For one unit (e.g., a device, a system, a component of adevice or system, combinations thereof, and/or the like) to be incommunication with another unit means that the one unit is able todirectly or indirectly receive information from and/or transmitinformation to the other unit. This may refer to a direct or indirectconnection (e.g., a direct communication connection, an indirectcommunication connection, and/or the like) that is wired and/or wirelessin nature. Additionally, two units may be in communication with eachother even though the information transmitted may be modified,processed, relayed, and/or routed between the first and second unit. Forexample, a first unit may be in communication with a second unit eventhough the first unit passively receives information and does notactively transmit information to the second unit. As another example, afirst unit may be in communication with a second unit if at least oneintermediary unit processes information received from the first unit andcommunicates the processed information to the second unit.

With reference to the figures, the present disclosure is directed to amounting device 310, 410, 610 for maintaining proper tight and rigidalignment between cameras of a vehicle 102 a, such as an autonomous orsemi-autonomous vehicle. The mounting device 310, 410, 610 can besecured or fixed to a portion of the exterior of the vehicle 102 a, suchas to a roof or another exterior portion of the vehicle 102 a. As usedherein, a “vehicle” refers to any moving form of conveyance that iscapable of carrying either one or more human occupants and/or cargo andis powered by any form of energy. The term “vehicle” includes, but isnot limited to, cars, trucks, vans, trains, autonomous vehicles,aircraft, water-going vessels, boats, airplanes, helicopters, and/oraerial drones. An “autonomous vehicle” is a vehicle having a processor,programming instructions, and drivetrain components that arecontrollable by the processor without requiring a human operator. Anautonomous vehicle may be “fully autonomous” in that it does not requirea human operator for most or all driving conditions and functions, or itmay be “semi-autonomous” in that a human operator may be required incertain conditions or for certain operations, or that a human operatormay override the vehicle's autonomous system and may take control of thevehicle. Exemplary autonomous vehicles, which can be used with themounting devices and image systems of the present disclosure, are shownin FIGS. 1A and 1B. A system architecture 200 for controlling thevehicle 102 a is shown in FIG. 2 .

The mounting device 310, 410, 610 of the present disclosure can beconfigured to maintain a proper alignment and spacing between two ormore cameras used to obtain images that can be processed to generate astereo image of a scene. In some examples, the mounting device 310, 410,610 can be configured to avoid camera misalignment caused by thermalexpansion of portions of the vehicle 102 a or the mounting device 310,410, 610. For example, the mounting device 310, 410, 610 can be formedfrom materials with a low coefficient of thermal expansion so thatspacing between the cameras is maintained even as temperature changes.

Also, the mounting device 310, 410, 610 can be configured such thatexpansion, movement, deformation, and/or vibration of portions of thevehicle 102 a, to which the mounting device 310, 410, 610 is connected,do not cause misalignment of the cameras or movement which would distortimages captured by the cameras. For example, effects of thermalexpansion and vibration can be minimized by reducing points of rigid orfixed contact or connection between the mounting device 310, 410, 610and the vehicle 102 a. Specifically, in some examples, the mountingdevice 310, 410, 610 can include only a single point of rigid contact orconnection with the vehicle 102 a, such as a bracket positioned at acenter of the mounting device 310, 410, 610. The bracket can bepositioned a substantially equal distance between a first end of themounting device 310, 410, 610 and a second end of the mounting device310, 410, 610. As used herein, a “point of rigid contact or connection”can refer to a portion of the mounting device 310, 410, 610 that isfixedly connected to the vehicle 102 a in a manner that limits rotationand/or movement of the mounting device 310, 410, 610 relative to thevehicle 102 a. In some examples, all other portions of the mountingdevice 310, 410, 610 can be free from rigid contact or connection withthe vehicle 102 a, such that the bracket forms the only point of rigidcontact or connection between the mounting device 310, 410, 610 and thevehicle 102 a.

The mounting devices 310, 410, 610 of the present disclosure can also beconfigured to absorb vibrations of the vehicle 102 a or at least tolimit movement of the mounting device 310, 410, 610 and the camera(s)attached thereto during normal operation of the vehicle 102 a. Inparticular, the mounting devices 310, 410, 610 can be configured toabsorb vibrations or limit vibrational movement of the mounting device310, 410, 610 so that images, which can be used to generate the stereoimages of a scene, can be captured by the cameras as the vehicle 102 atravels over rough roadways, potholes, cracks, bumps, and other commonroadway hazards and conditions that occur on public roadways. Forexample, the mounting device 310, 410, 610 can include points offlexible contact with the vehicle 102 a. The points of flexible contactmay allow the mounting device 310, 410, 610 to move in at least onedirection relative to the vehicle 102 a to account for forces caused byvehicle vibrations. The mounting device 310, 410, 610 can also includeshock absorbers or dampers, such as elastomeric disks, rings, orsleeves, for absorbing movement of the mounting device 310, 410, 610 atthe flexible contact points with the vehicle 102 a, further limitingvibration forces from the vehicle 102 a from causing misalignment of thecameras supported by the mounting device 310, 410, 610.

1. Autonomous vehicle system

FIGS. 1A and 1B illustrate an exemplary vehicle system 100 for thevehicle 102 a, which may include the mounting device 310, 410, 610 ofthe present disclosure. In particular, as previously described, themounting device 310, 410, 610 is configured to support optical, vision,and/or image sensors (e.g., cameras) of the vehicle 102 a to maintainspacing and alignment between cameras so that stereo orthree-dimensional images can be created. Improving a quality of stereoor three-dimensional images captured by the cameras improves imageprocessing, object detection, and navigation for the vehicle 102 a andvehicle systems 100.

FIG. 1A illustrates the vehicle system 100 with the vehicle 102 atraveling along a road. The vehicle 102 a is generally configured todetect objects in proximity to the vehicle 102 a. The objects caninclude, but are not limited to, another vehicle 102 b, a cyclist 114(such as a rider of a bicycle, electric scooter, motorcycle, or thelike) and/or a pedestrian 116. As illustrated in FIG. 1A, the vehicle102 a may include a sensor system 111, an on-board computing device 113,a communications interface 117, and a user interface 115. The vehicle102 a may further include certain components (as illustrated, forexample, in FIG. 2 ) included in vehicles, which may be controlled bythe on-board computing device 113 using a variety of communicationsignals and/or commands, such as, for example, acceleration signals orcommands, deceleration signals or commands, steering signals orcommands, and/or braking signals or commands.

The sensor system 111 may include one or more sensors that are coupledto and/or are included within the vehicle 102 a, as illustrated in FIG.2 . For example, such sensors may include, without limitation, a LiDARsystem, a radio detection and ranging (RADAR) system, a laser detectionand ranging (LADAR) system, a sound navigation and ranging (SONAR)system, one or more cameras or vision sensors (e.g., visible spectrumcameras, such as stereo cameras, infrared cameras, etc.), temperaturesensors, position sensors (e.g., global positioning system (GPS), etc.),location sensors, fuel sensors, motion sensors (e.g., inertialmeasurement units (IMU), etc.), humidity sensors, occupancy sensors, orthe like. In some examples, the sensor system 111 can be configured todirect a laser beam or light beam 104 towards object(s) in proximity tothe vehicle 102 a and measure reflected light 106 reflected from theobjects back towards the vehicle 102 a, as shown in FIG. 1A. The sensordata can include information that describes the location of objectswithin the surrounding environment of the vehicle 102 a, informationabout the environment itself, information about the motion of thevehicle 102 a, information about a route of the autonomous vehicle 102,or the like. As the vehicle 102 a travels over a surface, such as aroadway, at least some of the sensors may collect data pertaining to thesurface. It should be noted that the LiDAR systems for collecting datapertaining to the surface may be included in systems other than thevehicle 102 a such as, without limitation, other vehicles (autonomous ordriven), robots, satellites, etc.

A network 108 for communication with the vehicle 102 a can include oneor more wired or wireless networks. For example, the network 108 mayinclude a cellular network (e.g., a long-term evolution (LTE) network, acode division multiple access (CDMA) network, a 3G network, a 4Gnetwork, a 5G network, another type of next generation network, etc.).The network 108 may also include a public land mobile network (PLMN), alocal area network (LAN), a wide area network (WAN), a metropolitan areanetwork (MAN), a telephone network (e.g., the Public Switched TelephoneNetwork (PSTN)), a private network, an ad hoc network, an intranet, theInternet, a fiber optic-based network, a cloud computing network, and/orthe like, and/or a combination of these or other types of networks.

The vehicle 102 a may retrieve, receive, display, and edit informationgenerated from a local application or delivered via network 108 from aremote computing device 110 and/or database 112. For example, thedatabase 112 may be configured to store and supply raw data, indexeddata, structured data, map data, program instructions, or other dataconfigurations.

The communications interface 117 may be configured to allowcommunication between the vehicle 102 a and external systems, such as,for example, external devices, sensors, other vehicles, servers, datastores, and/or databases. The communications interface 117 may utilizeany known or hereafter known protocols, protection schemes, encodings,formats, packaging, etc., such as, without limitation, Wi-Fi, aninfrared link, Bluetooth®, etc. The user interface system 115 may bepart of peripheral devices implemented within the vehicle 102 aincluding, for example, a keyboard, a touch screen display device, amicrophone, and/or a speaker.

FIG. 1B illustrates an exterior of the vehicle 102 a including a sensorhousing or sensor frame 150, which can be used to support components ofthe sensor system 111. For example, multiple cameras can be enclosedwithin or mounted to the sensor housing or sensor frame 150. Some of thecameras may be mounted a fixed distance apart in order to generatestereo images of a scene. The sensor frame 150 can also include otherobject-detection sensors for detecting objects and/or the environmentsurrounding the vehicle 102 a.

As shown in FIG. 1B, the sensor housing or frame 150 can be positionedat any convenient location on the exterior of the vehicle 102 a. Forexample, as shown in FIG. 1B, the sensor frame 150 is positioned on aroof 154 of the vehicle 102 a. In other examples, the sensor frame 150and/or other structures for supporting the cameras can be positioned atmany other locations on either the exterior of the vehicle 102 a, suchas on a trunk 156 of the vehicle 102 a, or proximate to a front portionor grill 158 of the vehicle 102 a. In other examples, components of thesensor housing or frame 150 can be positioned inside a cabin 160 of thevehicle 102 a.

In some examples, the sensor housing or frame 150 includes multipleopenings or apertures 152 for the camera(s) positioned around the sensorhousing or frame 150. The camera(s) and apertures 152 can be oriented indifferent directions to provide a panoramic view (i.e., a view of from180 degrees to 360 degrees) of objects and/or the environmentsurrounding the vehicle 102 a. The mounting device 310, 410, 610 of thepresent disclosure can be configured to be secured to the sensor housingor frame 150 for supporting cameras in fixed positions and alignmentduring operation of the vehicle 102 a. In other examples, the mountingdevice 310, 410, 610 can be attached between the sensor housing or frame150 and another portion of the exterior of the vehicle 102 a, such asbetween the frame 150 and the roof 154 of the vehicle 102 a.

2. Autonomous or semi-autonomous vehicle system architecture

FIG. 2 illustrates an exemplary system architecture 200 for vehicle 102a, such as an autonomous or semi-autonomous vehicle, in accordance withaspects of the present disclosure. It is understood, however, that othertypes of vehicles are considered within the scope of the technologydescribed herein and may contain more or less elements as described inassociation with FIG. 2 . As a non-limiting example, an airborne vehiclemay exclude brake or gear controllers, but may include an altitudesensor. In another non-limiting example, a water-based vehicle mayinclude a depth sensor. One skilled in the art will appreciate thatother propulsion systems, sensors, and controllers may be included basedon a type of vehicle, as is known.

As shown in FIG. 2 , the system architecture 200 comprises an engine ormotor 202 and various sensors 204-218 for measuring various parametersof the vehicle. In gas-powered or hybrid vehicles having a fuel-poweredengine, the sensors may include, for example, an engine temperaturesensor 204, a battery voltage sensor 206, an engine Rotations Per Minute(“RPM”) sensor 208, and a throttle position sensor 210. If the vehicleis an electric or hybrid vehicle, then the vehicle may have an electricmotor, and accordingly includes sensors, such as a battery monitoringsystem 212 (to measure current, voltage, and/or temperature of thebattery), motor current 214 and voltage 216 sensors, and motor positionsensors 218, such as resolvers and encoders.

Operational parameter sensors that are common to both types of vehiclesinclude, for example: a position sensor 236, such as an accelerometer,gyroscope and/or inertial measurement unit; a speed sensor 238; and anodometer sensor 240. The vehicle 102 a also may have a clock 242 thatthe system architecture 200 uses to determine vehicle time duringoperation. The clock 242 may be encoded into the vehicle on-boardcomputing device 220 (which may be the same as on-board computing device113 of FIG. 1 ), it may be a separate device, or multiple clocks may beavailable.

The vehicle also includes various sensors that operate to gatherinformation about the environment in which the vehicle is traveling.These sensors may include, for example: a location sensor 260 (e.g., aGlobal Positioning System (“GPS”) device); object detection sensors suchas one or more vision sensors or cameras 262, such as cameras used forobtaining a stereo image of a scene; a LiDAR system 264; and/or a radarand/or a sonar system 266. The sensors also may include environmentalsensors 268, such as a precipitation sensor and/or ambient temperaturesensor. The object detection sensors may enable the vehicle to detectobjects that are within a given distance range of the vehicle 102 a inany direction, while the environmental sensors 268 collect data aboutenvironmental conditions within the vehicle's area of travel.

During operations, information is communicated from the sensors to avehicle on-board computing device 220. The on-board computing device 220may be implemented using a computer system, such as the computer system700 illustrated in FIG. 7 . The vehicle on-board computing device 220analyzes the data captured by the sensors and optionally controlsoperations of the vehicle 102 a based on results of the analysis. Forexample, the vehicle on-board computing device 220 may control: brakingvia a brake controller 222; direction via a steering controller 224;speed and acceleration via a throttle controller 226 (in a gas-poweredvehicle) or a motor speed controller 228 (such as a current levelcontroller in an electric vehicle); a differential gear controller 230(in vehicles with transmissions); and/or other controllers. Auxiliarydevice controller 254 may be configured to control one or more auxiliarydevices, such as testing systems, auxiliary sensors, or mobile devicestransported by the vehicle 102 a.

Geographic location information may be communicated from the locationsensor 260 to the on-board computing device 220, which may then access amap of the environment that corresponds to the location information todetermine known fixed features of the environment, such as streets,buildings, stop signs, and/or stop/go signals. Captured images from thecamera(s) 262 and/or object detection information captured from sensors,such as LiDAR system 264, is communicated from those sensors to theon-board computing device 220. The object detection information and/orcaptured images are processed by the on-board computing device 220 todetect objects in proximity to the vehicle 102 a. Any known or to beknown technique for making an object detection based on sensor dataand/or captured images can be used in the embodiments disclosed in thisdocument.

LiDAR information is communicated from LiDAR system 264 to the on-boardcomputing device 220. Additionally, captured images are communicatedfrom the camera(s) 262 to the vehicle on-board computing device 220. TheLiDAR information and/or captured images are processed by the vehicleon-board computing device 220 to detect objects in proximity to thevehicle 102 a. The manner in which the object detections are made by thevehicle on-board computing device 220 includes such capabilitiesdetailed in this disclosure.

The on-board computing device 220 may include and/or may be incommunication with a routing controller 231 that generates a navigationroute from a start position to a destination position for the vehicle102 a. The routing controller 231 may access a map data store toidentify possible routes and road segments that a vehicle can travel onto get from the start position to the destination position. The routingcontroller 231 may score the possible routes and identify a preferredroute to reach the destination. For example, the routing controller 231may generate a navigation route that minimizes Euclidean distancetraveled or other cost function during the route, and may further accessthe traffic information and/or estimates that can affect an amount oftime it will take to travel on a particular route. Depending onimplementation, the routing controller 231 may generate one or moreroutes using various routing methods, such as Dijkstra's algorithm,Bellman-Ford algorithm, or other algorithms. The routing controller 231may also use the traffic information to generate a navigation route thatreflects expected conditions of the route (e.g., current day of the weekor current time of day, etc.), such that a route generated for travelduring rush-hour may differ from a route generated for travel late atnight. The routing controller 231 may also generate more than onenavigation route to a destination and send more than one of thesenavigation routes to a user for selection by the user from among variouspossible routes.

In various examples, the on-board computing device 220 may determineperception information of the surrounding environment of the vehicle 102a. Based on the sensor data provided by one or more sensors and locationinformation that is obtained, the on-board computing device 220 maydetermine perception information of the surrounding environment of thevehicle 102 a. The perception information may represent what an ordinarydriver would perceive in the surrounding environment of the vehicle 102a. The perception data may include information relating to one or moreobjects in the environment of the vehicle 102 a. For example, theon-board computing device 220 may process sensor data (e.g., LiDAR orRADAR data, camera images, etc.) in order to identify objects and/orfeatures in the environment of the vehicle 102 a. The objects mayinclude traffic signals, road way boundaries, other vehicles,pedestrians, and/or obstacles, etc. The on-board computing device 220may use any known or hereafter known object recognition algorithms,video tracking algorithms, and computer vision algorithms (e.g., trackobjects frame-to-frame iteratively over a number of time periods) todetermine the perception.

In some examples, the on-board computing device 220 may also determine,for one or more identified objects in the environment, the current stateof the object. The state information may include, without limitation,for each object: current location; current speed and/or acceleration,current heading; current pose; current shape, size, or footprint; type(e.g., vehicle vs. pedestrian vs. bicycle vs. static object orobstacle); and/or other state information.

The on-board computing device 220 may perform one or more predictionand/or forecasting operations. For example, the on-board computingdevice 220 may predict future locations, trajectories, and/or actions ofone or more objects. For example, the on-board computing device 220 maypredict the future locations, trajectories, and/or actions of theobjects based at least in part on perception information (e.g., thestate data for each object comprising an estimated shape and posedetermined as discussed below), location information, sensor data,and/or any other data that describes the past and/or current state ofthe objects, the vehicle 102 a, the surrounding environment, and/ortheir relationship(s). For example, if an object is a vehicle and thecurrent driving environment includes an intersection, the on-boardcomputing device 220 may predict whether the object will likely movestraight forward or make a turn. If the perception data indicates thatthe intersection has no traffic light, the on-board computing device 220may also predict whether the vehicle may have to fully stop prior toenter the intersection.

In various embodiments, the on-board computing device 220 may determinea motion plan for the vehicle 102 a. For example, the on-board computingdevice 220 may determine a motion plan for the vehicle 102 a based onthe perception data and/or the prediction data. Specifically, givenpredictions about the future locations of proximate objects and otherperception data, the on-board computing device 220 can determine amotion plan for the vehicle 102 a that best navigates the vehicle 102 arelative to the objects at their future locations.

In some examples, the on-board computing device 220 may receivepredictions and make a decision regarding how to handle objects and/oractors in the environment of the vehicle 102 a. For example, for aparticular actor (e.g., a vehicle with a given speed, direction, turningangle, etc.), the on-board computing device 220 decides whether toovertake, yield, stop, and/or pass based on, for example, trafficconditions, map data, state of the autonomous vehicle, etc. Furthermore,the on-board computing device 220 also plans a path for the vehicle 102a to travel on a given route, as well as driving parameters (e.g.,distance, speed, and/or turning angle). That is, for a given object, theon-board computing device 220 decides what to do with the object anddetermines how to do it. For example, for a given object, the on-boardcomputing device 220 may decide to pass the object and may determinewhether to pass on the left side or right side of the object (includingmotion parameters such as speed). The on-board computing device 220 mayalso assess the risk of a collision between a detected object and thevehicle 102 a. If the risk exceeds an acceptable threshold, it maydetermine whether the collision can be avoided if the vehicle 102 afollows a defined vehicle trajectory and/or implements one or moredynamically generated emergency maneuvers performed in a pre-definedtime period (e.g., N milliseconds). If the collision can be avoided,then the on-board computing device 220 may execute one or more controlinstructions to perform a cautious maneuver (e.g., mildly slow down,accelerate, change lane, or swerve). In contrast, if the collisioncannot be avoided, then the on-board computing device 220 may executeone or more control instructions for execution of an emergency maneuver(e.g., brake and/or change direction of travel).

As discussed above, planning and control data regarding the movement ofthe vehicle 102 a is generated for execution. The on-board computingdevice 220 may, for example, control braking via a brake controller 222;direction via a steering controller 224; speed and acceleration via athrottle controller 226 (in a gas-powered vehicle) or a motor speedcontroller 228 (such as a current level controller in an electricvehicle); a differential gear controller 230 (in vehicles withtransmissions); and/or other controllers.

3. Mounting device for camera alignment

FIGS. 3A-3E illustrate an exemplary mounting device 310 configured to besecured to the exterior of the vehicle 102 a for supporting cameras,which may be the camera(s) discussed above in connection with the sensorsystem 111 or system architecture 200. FIG. 3A is a perspective view ofa front portion of the mounting device 310. FIG. 3B is a perspectiveview of a rear portion of the mounting device 310. FIG. 3C is a frontview of the mounting device 310. FIG. 3D is a side view of the mountingdevice 310. FIG. 3E is a front view of a camera mount 324, 326 of themounting device 310. The mounting device 310 can be secured to the roof154 of the vehicle 102 a and/or to the sensor housing or frame 150. Themounting device 310 can be configured to support a first camera 312 anda second camera 314 (shown in FIG. 3J) to maintain fixed and rigidalignment between the cameras 312, 314 to capture images that can beused to generate the stereo images of a scene.

The mounting device 310 comprises an elongated beam 316 for maintainingthe proper spacing and alignment between the first camera 312 and thesecond camera 314. FIGS. 3F and 3G show the elongated beam 316 separatedfrom other components of the mounting device 310. Specifically, FIG. 3Fis a perspective view of the elongated beam 316 of the mounting device310. FIG. 3G is a cross-sectional view of the elongated beam 316 of themounting device 310 taken along line 3G of FIG. 3F.

With specific reference to FIGS. 3A-3E, the elongated beam 316 includesa first end portion 318, a second end portion 320, and a side surface322 extending between the first end portion 318 and the second endportion 320. As used herein, a “beam” refers to a substantially rigidelongated member that does not deform or bend when exposed to normal oranticipated forces. For example, the elongated beam 316 of the presentdisclosure can be sufficiently rigid to avoid bending or flexing whenexposed to forces caused by vibration of portions of the vehicle 102 aas the vehicle 102 a drives on a road. The elongated beam 316 can have acircular shaped cross-section or other regular or irregular crosssection shapes, such as a square, rectangle, oval, or I-beam shape. Insome examples, as described in further detail herein, the elongated beam316 can be hollow, such as a hollow tube having cylindrical inner andouter surfaces. The elongated beam 316 is formed from a rigid materialwith low thermal expansion so that cameras mounted to the elongated beam316 do not become misaligned when the elongated beam 316 is exposed tohigh or low temperatures. In particular, the elongated beam 316desirably does not substantially expand or contract when exposed tonormal or expected temperatures to which the vehicle 102 a may beexposed during normal use. For example, the elongated beam 316 may beconfigured to avoid expansion or contract when exposed to temperaturesfrom varying extreme weather conditions (e.g., −35° F. to 122° F. (about−35° C. to 50° C.)). In some examples, the elongated beam 316 comprisescarbon fiber. For example, the elongated beam 316 can be a carbon fibertube produced by rolling, protrusion, or other common manufacturingmethods.

The mounting device 310 further comprises a first camera mount 324 onthe first end portion 318 of the elongated beam and a second cameramount 326 on the second end portion 320 of the elongated beam 316. Insome examples, the camera mounts 324, 326 can be integrally formed withthe elongated beam 316. For example, the elongated beam 316 can bemolded or formed with camera mounts 324, 326 extending from the firstend portion 318 and/or the second end portion 320 of the elongated beam316. Alternatively, the camera mounts 324, 326 can be separatestructures that are secured to the first end portion 318 and/or thesecond end portion 320 of the elongated beam 316 by adhesives,fasteners, clips, clamps, or other connectors as are known in the art.For example, as shown in FIG. 3E, the camera mounts 324, 326 can includea cylindrical connector portion 328 configured to be inserted in an openend of the tubular elongated beam 316. The camera mounts 324, 326 canalso include a flat, u-shaped supporting structure extending from theconnector portion 328. For example, the supporting structure can includea first side portion 330 proximate to the connector portion 328, asecond side portion 332 spaced apart from the connector portion 328, anda lower side portion 334 extending between the first side portion 330and the second side portion 332. The side portions 330, 332, 334 form agap or receiving space having a width D1 sized to receive cameras, suchas the cameras 312, 314 shown in FIG. 3J. The side portions 330, 332 caninclude openings 336 or through holes for securing one of the cameras312, 314 to the camera mounts 324, 326. For example, each side portion330, 332 can include three vertically aligned openings 336 for securingone of the cameras 312, 314 to the camera mount 324, 326. The cameramounts 324, 326 can be formed from any convenient material that can bemanufactured, machined, or molded to a desired final shape. For example,the camera mounts 324, 326 can be formed from metal, such as aluminum.The camera mounts 324, 326 can also include various other configurationsof clips, clamps, connectors, or anchors as are known in the artdepending upon the type of camera being used with the mounting device310.

With specific reference to FIGS. 3A-3C, as well as FIGS. 3H and 3I, themounting device 310 further comprises a bracket 338 positioned betweenthe first end portion 318 and the second end portion 320 of theelongated beam 316. FIGS. 3H and 3I show the bracket 338 separated fromother portions of the mounting device 310. Specifically, FIG. 3H is aperspective view of the bracket 338 of the mounting device 310. FIG. 3Iis a side view of the bracket 338 of the mounting device 310. In someexamples, the bracket 338 is configured to fixedly or rigidly connectthe elongated beam 316 to a portion of a vehicle, such as the vehicle102 a shown in FIGS. 1A and 1B. For example, the bracket 338 can beconfigured to secure the elongated beam 316 to an exterior portion orpanel of the vehicle 102 a, such as to the roof 154 of the vehicle 102a. The bracket 338 is configured to restrict rotation of the elongatedbeam 316 about multiple axes, such as restricting rotation in a pitchdirection (shown by arrow A1 in FIG. 3C) and a yaw direction (shown byarrow A2 in FIG. 3C) by a predetermined angular distance. For example,the bracket 338 can be configured to restrict rotation of the elongatedbeam 316 to a maximum pitch and maximum yaw of 0.5 milliradians (mrad).The bracket 338 can also be configured to restrict axial rotation (shownby arrow A3 in FIG. 3C) of the elongated beam 316 to ensure that afield-of-view of cameras, such as the cameras 312, 314 shown in FIG. 3J,is aimed in a correct or desired direction.

With specific reference to FIGS. 3F and 3G, as well as FIGS. 3A-3E, theelongated beam 316 of the mounting bracket 310 can be a tube, such as acylindrical tube, comprising a first open end 340, a second open end342, and a sidewall 344 extending therebetween. A cylindrical tube maybe preferred because cylindrical tubes are often easier to produce usingcommon carbon fiber processing methods than beams, elongated members, ortubes with other cross-sectional shapes. In other examples, aspreviously described, the elongated beam 316 or tube may have othercross-sectional shapes, such as a square, rectangle, oval, or I-beamshape.

A length of the elongated beam 316 is determined based on how far apartthe cameras, such as cameras 312, 314, need to be for the types ofstereo images being captured and may depend, for example, on aresolution, aperture size, or field-of-view dimensions of the cameras.For example, the elongated beam 316 can have an axial length L1 (shownin FIG. 3C) of from about 440 mm to about 540 mm. The elongated beam 316can have an outer diameter OD of about 60 mm to 100 mm and an innerdiameter ID of about 54 mm to 96 mm (shown in FIG. 3G). A thickness ofthe beam 316 can be selected in order to ensure that a moment of inertiaof the beam 316 is sufficient so that the beam 316 does not rotate dueto vibration forces or the weight of the cameras 312, 314 or cameramounts 324, 326. The beam 316 should also be sufficiently thick so thatthe beam 316 does not deform, bend, or flex when exposed to expected oranticipated forces from the vehicle 102 a. For example, a thickness ofthe beam 316 can be from about 2.0 mm to about 3.0 mm.

As previously described, the camera mounts 324, 326 can be connected tothe open first end 340 and the open second end 342 of the elongated beam316. For example, the cylindrical connectors 328 of the camera mounts324, 326 can be inserted into the open ends 340, 342 of the tubularelongated beam 316. In order to ensure proper fit, the cylindricalconnector 328 can have an outer diameter that substantially matches theinner diameter ID of the elongated beam 316 so that the connectorportion 328 can be fixedly and firmly secured to the elongated beam 316.The cylindrical connector portion 328 can be adhered to the innersurface of the elongated beam 316 by adhesives, welds, and/or mechanicalfasteners as are known in the art.

As previously described, the elongated beam 316 is configured to beconnected to the bracket 338 for securing the elongated beam 316 andcameras, such as the cameras 312, 314, to the vehicle 102 a. The bracket338 can be a molded structure formed from metal. In some examples, thebracket 338 can be formed from the same material as the material thatforms the portion of the vehicle 102 a to which the bracket 338 ismounted and can have similar or identical thermal expansion propertiesas the portion of the vehicle 102 a to which the bracket 338 is mounted.In some examples, the exterior of the vehicle 102 a and the bracket 338are formed from aluminum and/or from an aluminum alloy, which has acoefficient of thermal expansion of about 25.5° C⁻¹. As previouslydescribed, the elongated beam 316 is formed from a material, such ascarbon fiber, having a low coefficient of thermal expansion, so that thelength L1 of the beam 316 and distance between the camera mounts 324,326 does not change as temperature of the elongated beam 316 increases.The coefficient of thermal expansion for carbon fiber is generallyassumed to be near to zero.

With reference to FIGS. 3A-3D, as well as FIGS. 3H and 3I, the bracket338 can include one or more bases 346, 348, base portions, or connectingstructures configured to be attached to the vehicle 102 a, and a wall350 extending from the one or more bases 346, 348 for supporting theelongated beam 316. For example, the bracket 338 can include a first orbottom base 346, such as a flat rigid plate, including a top surface 352and a bottom surface 354 configured to be in contact with the exteriorof the vehicle 102 a. The wall 350 can extend substantially verticallyfrom the top surface 352 of the bottom base 346 in an upwards direction.For example, the wall 350 can extend perpendicularly from the topsurface 352 of the bottom base 346, as shown in FIGS. 3H and 3I. Thebracket 338 can also include a second or side base 348, such as a plate,clamp, connector or bracket, extending from a side or peripheral edge ofthe wall 350. The bottom base 346 and the side base 348 can includeopenings 356 or through holes extending through the bases 346, 348 sizedto receive fasteners, such as bolts, screws, nails, or posts, forsecuring the bottom base 346 and the side base 348 to the vehicle 102 a.For example, the bottom base 346 can include multiple openings 356 orthrough holes for receiving vertically arranged fasteners to secure thebottom base 346 to an exterior panel of the vehicle 102 a, such as tothe roof 154 of the vehicle 102 a. The side base 348 can include arectangular bracket or connector with openings 356 or through holesconfigured to receive horizontally oriented fasteners for securing theside base 348 to a rectangular beam of the sensor frame 150 of thevehicle 102 a.

The wall 350 of the bracket 338 is configured to receive the elongatedbeam 316 and to maintain positioning of the elongated beam 316 andcameras, such as the cameras 312, 314 shown in FIG. 3J, so that imagesfor generating the stereo images can be obtained from the cameras. Forexample, the wall 350 can include an opening 358 sized to receive acentral portion of the elongated beam 316. For a tubular elongated beam316, as shown in FIGS. 3A-3G, the opening 358 through the wall 350 is acircle having a diameter D2 (shown in FIG. 3I), which corresponds withthe outer diameter OD of the elongated beam 316. For example, thediameter D2 of the opening 358 can be about 60 mm to about 100 mm. Theelongated beam 316 can be fixedly secured to the opening 358 through thewall 350 by an adhesive, such as a curable epoxy. Many differentcommercially-available curable epoxy materials from differentmanufacturers can be used for securing the elongated beam 316 to thewall 350 of the bracket 338. For example, suitable epoxy adhesives thatcan be used with the mounting device 310 of the present disclosure aremanufactured by Olin Corporation, Huntsman Corporation, Dow Inc., andothers. In some examples, the adhesive includes or is mixed withspacers, such as glass beads. The spacers, such as the glass beads, areprovided to maintain spacing between the outer surface of the elongatedbeam 316 and the inner surface of the opening 358 of the wall 350.Maintaining spacing between the elongated beam 316 and the wall 350ensures that the elongated beam 316 is appropriately positioned in theopening 358 of the wall 350 with the adhesive well distributed aroundthe elongated beam 316.

Dimensions of the wall 350 are selected to ensure that the elongatedbeam 316 is held securely to restrict movement and rotation of theelongated beam 316 relative to the bracket 338. However, as previouslydescribed, an area of contact between the elongated beam 316 and thewall 350 of the bracket 338 is also intended to be as small as possibleto limit effects of thermal expansion of the vehicle 102 a and/orbracket 338. For example, the wall 350 can be from about 40 mm to about60 mm or, preferably about 50 mm, thick in order to properly support theelongated beam 316, while avoiding problems caused by the thermalexpansion of the vehicle 102 a and bracket 338.

FIGS. 4A and 4B illustrate effects of thermal expansion and vibrationforces on a mounting device 410, which can be similar or identical tothe previously described mounting device 310 shown in FIGS. 3A-3E.Specifically, FIG. 4A shows the mounting device 410 with a bracket 438connected to a vehicle 402 a, such as an autonomous or semi-autonomousvehicle, at a single rigid contact or connection point at the center ofthe mounting device 410 and without thermal expansion forces orvibration forces. By contrast, FIG. 4B shows the mounting device 410 andbracket 438 when thermal expansion forces and vibration forces areapplied to the mounting device 410.

As shown in FIG. 4B, thermal expansion forces may cause portions of thevehicle 402 a and bracket 438 to expand, as shown by arrows A4. However,due to the lower coefficient of thermal expansion of the elongated beam416, the elongated beam 416 does not substantially expand. If themounting device 410 were attached to the vehicle 402 a at multiple rigidor fixed contact or connection points, the thermal expansion of thevehicle 402 a would cause the different contact points between thevehicle 402 a and the mounting device 410 to move away from each otherexerting axial forces on the elongated beam 416 potentially causingmisalignment of cameras positioned on the ends of the beam 416. However,because the mounting device 410 is only attached to the vehicle 402 athrough a single rigid contact or connection point at the centralbracket 438 and because the area of contact between the bracket 438 andthe elongated beam 416 is as small as possible, the thermal expansion ofportions of the vehicle 402 a is not transferred to the mounting device410. Accordingly, the mounting device 410, which is only attached to thevehicle 402 a at the single rigid contact or connection point,substantially preserves alignment of the cameras by preventing thermalexpansion of the vehicle 402 a from exerting forces on the elongatedbeam 416.

The mounting device 410 in FIG. 4B is also exposed to vibrations. Thevibrations can cause ends of the elongated beam 416 of the mountingdevice 410 to move vertically, as shown by arrows A5 in FIG. 4B.However, a distance between cameras mounted at opposite ends of theelongated beam 416 desirably remains substantially constant meaning thatalignment of the cameras is generally preserved.

With reference to FIGS. 4C and 4D, as previously described, in someexamples, the mounting device 410 includes areas of flexible or movablecontact between the mounting device 410 and the vehicle 402 a. Forexample, as shown in FIGS. 4C and 4D, the mounting device 410 caninclude vibration dampers or vibration damper assemblies 460 positioned,for example, between a first end portion 418 and a second end portion420 of the elongated beam 416 and the vehicle 402 a. The vibrationdamper assemblies 460 can include a fastening member 462, such as a postor bolt, extending through the elongated beam 416. The fastening member462 can include a top end including a head portion 464 or flange and abottom end 466 that engages the vehicle 402 a for securing the fasteningmember 462 to the vehicle 402 a. As shown in FIGS. 4C and 4D, thefastening members 462 extend through openings or holes in the elongatedbeam 416. The elongated beam 416 can be configured to move or slidealong the fastening members 462 so that thermal expansion and vibrationforces from the vehicle 402 a are not exerted on the elongated beam 416.

The vibration assemblies 460 further include the vibration dampers, suchas sleeves 468, 470, which are secured or connected to the fasteningmembers 462 and surround the fastening members 462. In particular,vibration dampers, such as the sleeves 468, 470, can be secured to themounting device 410 and vehicle 402 a under compression in order toabsorb vibrations exerted by the vehicle 402 a to the mounting device410. The sleeves 468, 470 can be formed from synthetic or naturalelastomeric materials (e.g., polypropylene, polyethylene, silicone,synthetic rubber, or natural rubber (e.g., isoprene)), and can includean opening that receives the fastening member 462. In some examples, asshown in FIG. 4C and 4D, the vibration damper assemblies 460 can includeupper elastomeric sleeves 468 positioned between the head portion 464 ofthe fastening members 462 and the outer surface of the elongated beam416, and lower elastomeric sleeves 470 positioned between the outersurface of the elongated beam 416 and the vehicle 402 a. The sleeves468, 470 are positioned to absorb vibration forces exerted on theelongated beam 416 preventing the elongated beam 416 from moving up anddown relative to the fastening members 462 and/or vehicle 402 a.

FIG. 4C shows the mounting device 410 and the vehicle 402 a beforethermal expansion forces or vibration forces are applied to the mountingdevice 410. In FIG. 4D, thermal expansion forces (shown by arrow A4 inFIG. 4D) and vibration forces (shown by arrow A5 in FIG. 4D) are beingexerted on the mounting device 410. As previously described, themounting device 410 is secured to the vehicle 402 a by the bracket 438,which is a single point of rigid contact or connection between themounting device 410 and the vehicle 402 a. The mounting device 410 isalso flexibly connected to the vehicle 402 a by the vibration damperassemblies 460 at the first and second end portions 418, 420 of theelongated beam 416. Because the elongated beam 416 is not fixed to thefastening members 462 of the vibration damper assemblies 460, the beam416 is free to move relative to the fastening members 462, which reduceseffects of thermal expansion of the vehicle 402 a and bracket 438. InFIG. 4D, the mounting device 410 is also exposed to vibration forces.However, unlike in FIG. 4B in which end portions 418, 420 of theelongated beam 416 moved vertically (shown by arrow A5 in FIG. 4B), theelongated beam in FIG. 4D remains substantially stationary because anyvertical forces exerted on the elongated beam 416 are absorbed by theelastomeric sleeves 468, 470 of the vibration damper assemblies 460connected to the end portions 418, 420 of the elongated beam 416.

4. Assembly method for a mounting device

FIG. 5 shows a flow chart including steps for assembling a mountingdevice, such as the previously described mounting device 310, and forsecuring the mounting device to a vehicle 102 a (shown in FIGS. 1A and1B), such as an autonomous or semi-autonomous vehicle. At step 510, aninstaller, such as a manufacturer, vehicle technician, or mechanic,obtains components of the mounting device 310. At step 512, theinstaller connects the cameras mount 324, 326 to the elongated beam 316.For example, the installer may insert the connection portions 328 of thecamera mounts 324, 326 through the open ends 340, 342 of the beam 316.The connection portions 328 may be secured in the beam 316 by anadhesive or fasteners. Once the camera mounts 324, 326 are in place, atstep 514, the installer inserts the elongated beam 316 of the mountingdevice 310 through the opening 358 in the wall 350 of the bracket 338,such that a portion of the elongated beam 316 between the first endportion 318 and the second end portion 320 of the elongated beam 316 isretained in the opening 358 of the wall 350. For example, as previouslydescribed, the bracket 338 can be positioned at a center of theelongated beam 316 so that the beam 316 and cameras, such as the cameras312, 314, connected thereto are balanced about or relative to thecentral bracket 338.

At step 516, in some examples, the installer can apply an adhesive, suchas a curable epoxy resin, between an outer surface of the elongated beam316 and an inner surface of the wall 350 of the bracket 338. Forexample, the installer may dispense flowable adhesive to a gap betweenthe outer surface of the beam 316 and the wall 350 from a suitablecontainer or packaging. Once the adhesive cures, the adhesive fixes thebracket 338 at the central position on the elongated beam 316.

At step 518, once the beam 316 is secured to the bracket 338, theinstaller attaches the bracket 338 of the mounting device 310 to avehicle, such as the vehicle 102 a shown in FIGS. 1A and 1B, by securingone or more fasteners through openings in the bases 346, 348 of thebracket 338 to fixedly connect the bracket 338 to a portion of thevehicle 102 a. For example, as previously described, the bottom base 346can be positioned in contact with a panel of the vehicle 102 a, such asthe vehicle roof 154 shown in FIG. 1B. The side base 348 can be securedto a portion of the sensor frame 150 shown in FIG. 1B. For example, theside base 348 can be secured to a rectangular beam or connector of thesensor housing or frame 150.

At step 520, the installer can also attach the cameras 312, 314 to thecamera mount 324, 326 by, for example, inserting fasteners, such asbolts or screws, through the openings 336 on the side portions 330, 332of the camera mounts 324, 326. The installer may also attach variouspower and/or data transmission wires or cables to the cameras 312, 314to provide power for the cameras 312, 314 and to place the cameras 612,614 in communication with control and/or data processing and collectionsystems of the vehicle 102 a. The above steps need not be performed inthe order recited above. For example, step 518 (attaching the bracket tothe vehicle) can occur before step 512. By way of further example, step512 can follow step 514 and/or step 516.

At step 522, once the cameras 312, 314 are secured to the camera mount324, 326, the cameras 312, 314 can be calibrated in order to prepare thecameras 312, 314 for use. As previously described, the mounting device310 is configured to stabilize and maintain alignment between thecameras 312, 314 so that stereo images of the scene can be obtained bythe cameras 312, 314. Therefore, it is expected that the cameras 312,314 will not need to be recalibrated often because the mounting device310 maintains the fixed distance between and proper alignment of thecameras 312, 314.

5. Stereo image generation system and computer control system

FIG. 6 shows a system 680 for generating stereo images from the cameras612, 614, which can be similar or identical to the previously describedcameras 312, 314 shown in FIG. 3J. As in previous embodiments, thecameras 612, 614 are connected to and supported by a mounting device610, which can be similar or identical to the previously describedmounting devices 310, 410. For example, the first camera 612 can beconnected to a first camera mount 624 of the mounting device 610 and thesecond camera 614 can be connected to a second camera mount 626 of themounting device 610. The system 680 further comprises a controller orprocessor 682, such as a computer processor of the on-board computingdevice 220 (shown in FIG. 2 ), and computer memory 684 comprisinginstructions for obtaining, processing, storing, and transmitting imagescaptured by the cameras 612, 614. The controller or processor 682 is inelectrical communication with the first camera 612 and the second camera614. The controller or processor 682 is configured to receive andprocess pairs of images substantially simultaneously captured by thefirst camera 612 and the second camera 614. Based on the receivedimages, the controller or processor 682 is configured to generate stereoimages of objects 686 in a scene from the pairs of images. As previouslydescribed, the generated stereo images can be analyzed by the processor682 and/or by other computing devices or systems to determineinformation about the scene including, for example, a distance betweenobjects 686 in the scene and a distance between objects 686 and thecameras 612, 614. The determined distance or depth information alongwith information from other sensors of a vehicle perception system canbe used to identify certain objects 686 in a scene and/or to controlnavigation of the vehicle.

The on-board computing device 220 (shown in FIG. 2 ) of the vehicle 102a can be implemented using a computer system, such as the exemplarycomputer system 700 shown in FIG. 7 . The computer system 700 can be anycomputer capable of performing the functions described herein. Withreference to FIG. 7 , the computer system 700 includes one or moreprocessors (also called central processing units, or CPUs), such as aprocessor 704. The processor 704 is connected to a communicationinfrastructure or bus 706.

One or more processors 704 may each be a graphics processing unit (GPU).In an embodiment, a GPU is a processor that is a specialized electroniccircuit designed to process mathematically intensive applications. TheGPU may have a parallel structure that is efficient for parallelprocessing of large blocks of data, such as mathematically intensivedata common to computer graphics applications, images, videos, etc.

The computer system 700 also includes user input/output device(s) 703,such as monitors, keyboards, pointing devices, etc., that communicatewith communication infrastructure 706 through user input/outputinterface(s) 703.

The computer system 700 also includes a main or primary memory 708, suchas random access memory (RAM). The main memory 708 may include one ormore levels of cache. The main memory 708 has stored therein controllogic (i.e., computer software) and/or data.

The computer system 700 may also include one or more secondary storagedevices or memory 710. Secondary memory 710 may include, for example, ahard disk drive 712 and/or a removable storage device or drive 714.Removable storage drive 714 may be a floppy disk drive, a magnetic tapedrive, a compact disk drive, an optical storage device, tape backupdevice, and/or any other storage device/drive.

Removable storage drive 714 may interact with a removable storage unit718. Removable storage unit 718 includes a computer usable or readablestorage device having stored thereon computer software (control logic)and/or data. Removable storage unit 718 may be a floppy disk, magnetictape, compact disk, DVD, optical storage disk, and/any other computerdata storage device. Removable storage drive 714 reads from and/orwrites to removable storage unit 818 in a well-known manner.

According to an exemplary embodiment, secondary memory 710 may includeother means, instrumentalities, or other approaches for allowingcomputer programs and/or other instructions and/or data to be accessedby computer system 700. Such means, instrumentalities, or otherapproaches may include, for example, a removable storage unit 722 and aninterface 720. Examples of the removable storage unit 722 and theinterface 720 may include a program cartridge and cartridge interface(such as that found in video game devices), a removable memory chip(such as an EPROM or PROM) and associated socket, a memory stick and USBport, a memory card and associated memory card slot, and/or any otherremovable storage unit and associated interface.

The computer system 700 may further include a communication or networkinterface 724. The communication interface 724 enables computer system700 to communicate and interact with any combination of remote devices,remote networks, remote entities, etc. (individually and collectivelyreferenced by reference number 728). For example, communicationinterface 724 may allow the computer system 700 to communicate withremote devices 728 over communications path 726, which may be wiredand/or wireless, and which may include any combination of LANs, WANs,the Internet, etc. The control logic and/or data may be transmitted toand from computer system 700 via communication path 726.

In some examples, a tangible, non-transitory apparatus or article ofmanufacture comprising a tangible, non-transitory computer useable orreadable medium having control logic (software) stored thereon is alsoreferred to herein as a computer program product or program storagedevice. This includes, but is not limited to, computer system 700, mainmemory 708, secondary memory 710, and removable storage units 718 and722, as well as tangible articles of manufacture embodying anycombination of the foregoing. Such control logic, when executed by oneor more data processing devices (such as computer system 700), causessuch data processing devices to operate as described herein.

Based on the teachings contained in this disclosure, it will be apparentto persons skilled in the relevant art(s) how to make and useembodiments of this aspect of the disclosure using data processingdevices, computer systems, and/or computer architectures other than thatshown in FIG. 7 . In particular, embodiments can operate with software,hardware, and/or operating system implementations other than thosedescribed herein.

It is to be appreciated that the Detailed Description section, and notany other section, is intended to be used to interpret the claims. Othersections can set forth one or more but not all exemplary embodiments ascontemplated by the inventor(s), and thus, are not intended to limitthis disclosure or the appended claims in any way.

While this disclosure describes exemplary embodiments for exemplaryfields and applications, it should be understood that the disclosure isnot limited thereto. Other embodiments and modifications thereto arepossible, and are within the scope and spirit of this disclosure.

Embodiments have been described herein with the aid of functionalbuilding blocks illustrating the implementation of specified functionsand relationships thereof. The boundaries of these functional buildingblocks have been arbitrarily defined herein for the convenience of thedescription. Alternate boundaries can be defined as long as thespecified functions and relationships (or equivalents thereof) areappropriately performed. Also, alternative embodiments can performfunctional blocks, steps, operations, methods, etc. using orderingsdifferent than those described herein.

References herein to “one embodiment,” “an embodiment,” “an exampleembodiment,” or similar phrases, indicate that the embodiment describedcan include a particular feature, structure, or characteristic, butevery embodiment can not necessarily include the particular feature,structure, or characteristic. Moreover, such phrases are not necessarilyreferring to the same embodiment. Further, when a particular feature,structure, or characteristic is described in connection with anembodiment, it would be within the knowledge of persons skilled in therelevant art(s) to incorporate such feature, structure, orcharacteristic into other embodiments whether or not explicitlymentioned or described herein. Additionally, some embodiments can bedescribed using the expression “coupled” and “connected” along withtheir derivatives. These terms are not necessarily intended as synonymsfor each other. For example, some embodiments can be described using theterms “connected” and/or “coupled” to indicate that two or more elementsare in direct physical or electrical contact with each other. The term“coupled,” however, can also mean that two or more elements are not indirect contact with each other, but yet still co-operate or interactwith each other.

The breadth and scope of this disclosure should not be limited by any ofthe above-described exemplary embodiments, but should be defined only inaccordance with the following claims and their equivalents.

What is claimed is:
 1. A mounting device, comprising: an elongated beamcomprising a first end portion, a second end portion, and a side surfaceextending between the first end portion and the second end portion; afirst camera mount attached to the first end portion configured tosupport a first camera; a second camera mount attached to the second endportion configured to support a second camera; a bracket positionedbetween the first end portion and the second end portion for fixedlyconnecting the elongated beam to a vehicle, the bracket comprising atleast one base configured to be attached to the vehicle and a wallextending from the at least one base comprising an opening sized toreceive the elongated beam, such that engagement between the wall andthe elongated beam restricts rotation of the elongated beam aboutmultiple axes; a first fastening member extending through the first endportion of the elongated beam that secures the first end portion to thevehicle; and a second fastening member extending through the second endportion of the elongated beam that secures the second end portion to thevehicle; wherein the first fastening member and the second fasteningmember each comprises a first end comprising a head, a second endengaged to the vehicle, an upper elastomeric sleeve positioned betweenthe head and the elongated beam, and a lower elastomeric sleevepositioned between the elongated beam and the vehicle.
 2. The mountingdevice of claim 1, wherein the bracket is positioned a substantiallyequal distance between the first end portion and the second end portion.3. The mounting device of claim 1, wherein the bracket restrictsrotation of the elongated beam in pitch and yaw directions.
 4. Themounting device of claim 3, wherein the bracket further restricts axialrotation of the elongated beam.
 5. The mounting device of claim 1,wherein at least a portion of the elongated beam extending through theopening of the wall comprises a hollow cylindrical tube.
 6. The mountingdevice of claim 1, wherein the elongated beam comprises a hollowcylindrical tube comprising a sidewall with a thickness of from about2.0 mm to about 3.0 mm.
 7. The mounting device of claim 1, wherein theelongated beam comprises carbon fiber and the bracket comprises aluminumor an aluminum alloy; and wherein the bracket is configured to besecured to a portion of the vehicle comprising aluminum or an aluminumalloy.
 8. The mounting device of claim 1, wherein the elongated beamcomprises a first material and the bracket comprises a second material,wherein a coefficient of thermal expansion (CTE) of the first materialis less than the CTE of the second material.
 9. The mounting device ofclaim 1, further comprising an adhesive that adheres the side surface ofthe elongated beam to an inner surface of the wall; wherein the adhesivecomprises a cured epoxy and a plurality of spacers embedded in the curedepoxy for maintaining substantially equal spacing between the sidesurface of the beam and the inner surface of the wall.
 10. The mountingdevice of claim 9, wherein the plurality of spacers comprise glassbeads.
 11. The mounting device of claim 1, wherein the at least one basecomprises a plurality of openings, each of which is configured toreceive a fastener to attach the at least one base to the vehicle. 12.The mounting device of claim 1, wherein the at least one base comprisesa first base portion configured to be attached to a first portion of thevehicle and a second base portion extending from the first base portionconfigured to be attached to a second portion of the vehicle.
 13. Themounting device of claim 1, wherein at least the lower elastomericsleeves are under compression in order to absorb vibrations exerted bythe vehicle to the mounting device.
 14. A mounting device, comprising:an elongated beam comprising a first end portion, a second end portion,and a side surface extending between the first end portion and thesecond end portion; a first camera mount attached to the first endportion configured to support a first camera; a second camera mountattached to the second end portion configured to support a secondcamera; and a bracket positioned between the first end portion and thesecond end portion for fixedly connecting the elongated beam to avehicle, the bracket comprising at least one base configured to beattached to the vehicle and a wall extending from the at least one basecomprising an opening sized to receive the elongated beam, such thatengagement between the wall and the elongated beam restricts rotation ofthe elongated beam about multiple axes, wherein the at least one basecomprises a first base portion configured to be mounted to an exteriorsurface of a roof of the vehicle and a second base portion configured tobe mounted to a sensor frame positioned on the roof of the vehicle. 15.The mounting device of claim 14, further comprising one or morevibration dampers positioned between at least one of the first endportion and the vehicle and the second end portion and the vehicle. 16.The mounting device of claim 15, further comprising at least onefastening member extending from one of the first and second end portionsto the vehicle, wherein the one or more vibration dampers compriseelastomeric sleeves extending around the at least one fastening member.17. A system, comprising: a mounting device comprising: an elongatedbeam comprising a first end portion, a second end portion, and a sidesurface extending between the first end portion and the second endportion; a first camera mount attached to the first end portionconfigured to support a first camera; a second camera mount attached tothe second end portion configured to support a second camera; and abracket positioned between the first end portion and the second endportion for fixedly connecting the elongated beam to a vehicle, thebracket comprising at least one base configured to be attached to thevehicle and a wall extending from the at least one base comprising anopening sized to receive the elongated beam, such that engagementbetween the wall and the elongated beam restricts rotation of theelongated beam about multiple axes, wherein the mounting device isfixedly connected to an exterior of a vehicle body by one or morefasteners extending through openings of the at least one base of thebracket, wherein the elongated beam comprises a first material and thebracket comprises a second material, and wherein a coefficient ofthermal expansion (CTE) of the first material is less than the CTE ofthe second material; a first camera attached to the first camera mount;a second camera attached to the second camera mount; and at least oneprocessor in electrical communication with the first camera and thesecond camera, wherein the at least one processor is configured togenerate at least one stereo image of a scene based on a first imagereceived from the first camera and a second image received from thesecond camera, wherein the first image and the second image are acquiredsubstantially simultaneously.
 18. A mounting device, comprising: anelongated beam formed from a first material and comprising a first endportion, a second end portion, and a side surface extending between thefirst end portion and the second end portion; a first camera mountattached to the first end portion configured to support a first camera;a second camera mount attached to the second end portion configured tosupport a second camera; and a bracket formed from a second materialpositioned between the first end portion and the second end portion forfixedly connecting the elongated beam to a vehicle, the bracketcomprising at least one base configured to be attached to the vehicleand a wall extending from the at least one base comprising an openingsized to receive the elongated beam; wherein the side surface of theelongated beam is adhered to an inner surface of the wall by anadhesive, the adhesive comprising a cured epoxy and a plurality ofspacers embedded in the cured epoxy for maintaining substantially equalspacing between the side surface of the beam and the inner surface ofthe wall.